Sound-creation interface

ABSTRACT

A sound-creation interface is used to create a sound-creation instrument such as a musical instrument. The musical instrument includes a mechanical-human interface such as a set of keys, strings or breathing pipe for user actuation and control of the system, a mechanical-electrical interface receiving inputs from the mechanical-human interface and for converting those inputs to a machine comprehensible signal and an electrical/processor interface receiving signals from the mechanical electrical interface and converting those signals into a processor comprehensible form. A processor receives processor comprehensible signals from the electrical-processor interface and includes a processor component and a memory component. The system allows simplified interaction between the user using the mechanical interface and the processor providing an improved sound-creation interface.

FIELD OF THE INVENTION

The present invention relates to a method and apparatus for detectingmovement of a user element, a sensor, a user input element, a userinput, a sound-creation instrument, a sound-creation interface, and alsoto a user interface, a sound-creation system, and a musical instrument.

BACKGROUND OF THE INVENTION

Musicians have been creating and performing electronic music for manyyears using computers and synthesizers. A musician inputs his or herinstructions into the computer or synthesizer using a computer keyboardand/or a piano-type keyboard having a set of pressable keys. However,whilst a wide range and variety of sounds can be created using acomputer or synthesizer, the output is necessarily limited in certainrespects as it is difficult to reproduce the complex sound of asaxophone being played or a guitar being strummed using distinct keypresses of a keyboard. Furthermore, live performances of electronicallygenerated music have in the past necessarily involved the musicianshaving to stand behind a computer or a keyboard on stage whilst themusician programs the computer and/or plays the keyboard. This can bedull viewing for the audience in comparison to a live rock band, inwhich many of the instruments are portable and can be worn and played bya musician moving about the stage.

Some of the issues regarding electronic music creation are addressed atleast to some extent with the provision of a sound creation interfacethat in one embodiment is used to create a sound-creation instrumentsuch as a musical instrument.

STATEMENT OF INVENTION

According to an embodiment, there is provided a method of detectingmovement of a user element having a shutter component moveable therewithinto a light path between an emitter and a detector comprisingreceiving, at the detector, light from the emitter, and monitoring forvariation in the amount of received light.

According to an embodiment, there is provided a sensor for detectingmovement of a user input element, the sensor comprising a light emitter,a light detector and a shutter component arranged to be in operableconnection with the user element and to be moveable therewith inresponse to user actuation of the user input element to at leastpartially occlude a light path between the light emitter and lightdetector, whereby a level of light detected at the light detectorcorresponds to a user actuation of the user input element.

According to an embodiment, there is provided a user input for asound-creation interface, the user input comprising a housing definingan airflow cavity therethrough, the airflow cavity having an entranceand an exit, a pressure chamber in fluid communication with the airflowcavity, a pressure tube in fluid communication with the pressure chamberand a variable bypass valve arranged at the entrance or exit of theairflow cavity to vary a rate of airflow through the airflow cavity.Spittle and condensation generated during use of the user input arevented through an aperture of the bypass valve.

According to another embodiment, there is provided a sound-creationinstrument comprising an attachment device for attaching the musicalinstrument to a user wearable strap, the attachment device being locatedat the center of mass of the sound-creation instrument.

According to a further embodiment there is provided a musical instrumentcomprising a plurality of user actuable inputs for creating musicalsounds and having a physical characteristic, the instrument furthercomprising a control element sharing said physical characteristic and asensor arranged to sense control element behavior.

According to an embodiment, there is provided a sound creation interfacearranged to receive a plurality of distinguishable user inputs andarranged to provide a sound-creation output from a plurality ofcandidate sound-creation outputs, the interface being programmable via auser input to assign a candidate sound-creation output to any one of theplurality of user inputs. As a result, a user input such as a useractuable key on a sound creation interface such as an electronic musicalinstrument, can be used to both program the instrument and perform.

According to another embodiment, there is provided a sound creationsystem including user viewable sound creation agents.

According to a further embodiment there is provided a sound creationsystem controlled according to program commands actuated by user keys.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other embodiments will be described, by way of example, withreference to the accompanying drawings, in which:

FIG. 1 is an overview of a sound-creation interface;

FIGS. 2 a, 2 b and 2 c are an overview of a mechanical-human interfaceof the sound-creation interface;

FIG. 3 is a schematic cross section of an embodiment of a key assembly;

FIG. 4 is an exploded view of the key assembly of FIG. 3;

FIG. 5 a is a perspective view of an embodiment of a prismatic elementof an optical sensor;

FIG. 5 b is a schematic cross section of the prismatic element of FIG. 5a;

FIG. 5 c is a zoomed in view of the cross section of FIG. 5 b, showingan example of a light path from an LED to a photodetector;

FIG. 5 d is a plan view of the prismatic element of FIG. 5 a;

FIG. 6 a is a perspective view of an embodiment of a shutter componentand a spring element;

FIG. 6 b is a cross section of the shutter component and spring elementof FIG. 6 a;

FIG. 6 c is a view of the underside of the shutter component and springelement of FIG. 6 a;

FIG. 6 d is a view of the flattened profile of the spring element ofFIG. 6 a prior to the spring being formed into shape;

FIG. 7 is a schematic diagram of the operation of the key scanningelectronics for two keys within a strip;

FIG. 8 is a representation of the digital control sequence used to scanstrips of keys (in this example, two keys of a strip of four keys);

FIG. 9 shows the control signal used to switch on an LED (top signal)along with the amplified and filtered signal generated by thephotodetector as a result of an LED being lit (bottom signal);

FIG. 9 a (top signal) shows a sequence of photo detector signalsgenerated as a result of a sequence of LEDs being lit in turn. Thebottom signal represents a control signal used to ‘reset’ the photodetector output back to zero before the next LED in the sequence is lit;

FIG. 9 b is a schematic view showing use of filters in the signalprocessing;

FIG. 9 c is a schematic view of a state diagram corresponding to FIG. 9b;

FIG. 10 a shows an embodiment of a breath controller;

FIG. 10 b is an exploded view of the components of the breath controllerof FIG. 10 a;

FIG. 10 c is a perspective view of a section through the valve body ofthe breath controller of FIGS. 10 a and 10 b;

FIG. 11 a is a front view of a schematic of a user wearing asound-creation instrument showing an embodiment of a strap andattachment means;

FIG. 11 b is a front view of a schematic of a user showing the strap andattachment means of FIG. 11 a;

FIG. 11 c is a rear view of a schematic of a user showing the strap ofFIG. 11 a;

FIG. 11 d is a perspective view of a schematic of a user wearing thestrap and attachment means of FIG. 11 a;

FIG. 11 e is a schematic representation of the attachment means of FIG.11 a, shown recessed into a base of the sound creation instrument;

FIG. 11 f is a section through the attachment means of FIG. 11 a;

FIG. 11 g is a schematic representation of an instrument attachmentdevice, balljoint mechanism and adjustment screw of the attachment meansof FIG. 11 a;

FIG. 11 h is a perspective detail view of a section through aninstrument attachment device and instrument mounting bracket;

FIG. 11 i is a perspective view of a spring of the instrument attachmentdevice of FIG. 11 h;

FIG. 12 is a schematic representation of a Graphical User Interface ofthe sound-creation interface;

FIG. 13 is a schematic representation of an individual component oragent of the sound-creation interface;

FIG. 14 a is a schematic diagram showing an operational tree structureof an agent;

FIG. 14 b shows a graphical representation of agent connections;

FIG. 15 is an embodiment of a flow diagram of use of the sound-creationinterface; and

FIG. 16 is an embodiment of a control mode of operation of thesound-creation interface;

FIG. 17 a is a perspective view of a portion of an over-frame for thesound-creation instrument;

FIG. 17 b is an exploded view of the over-frame for the sound-creationinstrument.

FIG. 18 is a flow diagram illustrating operation of a language agent;and

FIG. 19 shows phrases including deferred execution objects.

OVERVIEW

In overview a sound-creation interface is used to create asound-creation instrument such as a musical instrument. Referringfirstly to FIG. 1, the musical instrument is designated generally 100and includes a mechanical-human interface 102 such as a set of keys,strings or breathing pipe for user actuation and control of the system,a mechanical-electrical interface 104 receiving inputs from themechanical-human interface 102 and for converting those inputs to amachine comprehensible signal and an electrical/processor interfacereceiving signals from the mechanical electrical interface andconverting those signals into a processor comprehensible form 106. Aprocessor 108 receives processor comprehensible signals from theelectrical-processor interface 106 and includes a processor component110 and a memory component 112. The system allows simplified interactionbetween the user using the mechanical interface and the processorproviding an improved sound-creation interface.

Referring now to FIG. 2 a, a mechanical-human interface comprises in oneembodiment a plurality of user actuatable keys 125 and breath pipe 130.The keys 125 are arranged in an array for ease of user actuation and mayfor example be set out in an arrangement similar to that of the stringsof a guitar, with keys placed at fretpositions of the “strings”. Thekeys may be actuated or played by applying downward pressure in thecenter of the key or the key may be moved into a roll or yaw position bypressing in a specific direction on an outer part of the key. In thismanner, distinguishable user inputs can be played that are converted viathe human/mechanical interface and the mechanical/electrical interfaceinto machine comprehensible signals.

The breath pipe 130 is a separate form of user input into which a usermay blow, suck or hum in order to produce a measurable static pressuresignal that is then processed by a low pass filter at themechanical-electrical interface to produce an electrical signalcorresponding to the positive or negative static pressure exerted by theuser blowing, sucking or humming into the breath pipe. The breath pipecan be used by the musician to modulate the pitch of a note played. Thehumming input can be processed in one of two ways to achieve differentoutputs. Firstly, the humming can be used as an audio signal to modifyother audio signals created e.g., by using the keys or as an audiosignal in itself Secondly, the signal can be subjected to a frequencyanalysis using a Fast Fourier Transform (FFT). The derived signal canthen be used to produce user performance data that can be connectedwherever the user wishes in the system. The performance data derivedfrom the frequency and amplitude of a user's humming may, for example,be used to control the pitch of an audio oscillator or the frequency andQ (Quality Factor) of an audio filter. A further user input into thesound-creation instrument is a strip controller 132 shown schematicallyin FIG. 2 a. The strip controller 132 essentially comprises a variableresistor that a user may actuate with his/her fingers to produce aglissando effect to a note being played. Such strip controllers arecommercially available and as such will not be described in detail here.

In overview, a mechanical-electrical interface 104, shown in FIGS. 2 band 2 c, comprises for each key 125 an optical sensor 140 that directlyuses the level and direction of user input of the key 125 for conversionvia a printed circuit board 150 into an electrical signal. The opticalsensor 140 for each key comprises a plurality of light-emitting diodes(LEDs) 160 arranged around a central photo detector 170 on the printedcircuit board as seen schematically in FIGS. 2 a, 2 b and 2 c. Theoptical sensor 140 further comprises a prismatic element 180, seen inFIGS. 3 and 4 that sits over the LEDs and photo detector and is formedto receive light emitted from each of the LEDs to focus it onto thephoto detector. The user input element or key 125 sits directly over theoptical sensor and has attached to its underside a shutter component290. The shutter component is supported by a resilient biasing elementor spring 200 such that it is able to move in any of three axes ofmotion according to the specific user actuation of the key 125. Theshutter component sits over the prismatic element and as the user playsthe key, the shutter component moves downwards towards the prismaticelement to at least partially occlude or obstruct a light path betweenthe LEDs and the photo detector. In use of the sound-creation interface,the LEDs are cycled to continually sequentially emit light. A key pressor stroke actuates the shutter components to move into the light path ofthe LEDs to the prismatic element and blocks out light at least to someextent from one or more of the LEDs. The level of light reaching thephoto detector from each of the LEDs is converted into a voltage valuethat corresponds to the key actuation.

In overview, when the LEDs for a single detector cycle their lightemission, it is then simply necessary to synchronize detection at thedetector with each of the LEDs to obtain the corresponding light value.As discussed in more detail below where there are multiple detectorseach with respective multiple LEDs then detection additionally cycledbetween the multiple detectors and in particular a detection value issampled during a “detection-on” phase. By overlapping the “detection-on”phase by temporally adjacent detectors in the cycle increased cyclingfrequency can be obtained. By driving the signal value at the detectorto zero after sampling during the detection-on phase the value is resetor “crowbarred” again allowing fast sampling and optimal filteringwithout the reading of the light level from one LED affecting thereading of the next LED. Yet further movement of the shutter componentassociated with the user elements such as a key is simply detected bymonitoring for variation of the amount of received light and the degreeof movement can be assessed either of a single or multiple shuttersassociated with the key.

The interaction between the keys 125 with a processor, via theinstrument electronics, allows a sound-creation interface to be createdwhere distinguishable user inputs such as different pressures on thekeys or different roll and yaw values allow control of the creation ofmusic and programming of the system. For example the processor can storemultiple candidate sound-creation outputs such as different note pitch,length or synthesizer values and assign the output to any desired key orcombination of keys meaning that complete flexibility in music creationis provided. Yet further the user keys can in themselves be used toprogram the interface as well as subsequently to perform the musicaccording to their assigned values.

Importantly the system can be configured and controlled using a naturallanguage interface where words can be composed into human comprehensiblephrases or sentences. Furthermore, as described in more detail below,the language can be communicated as note sequences encoding the words ina musical scale.

As a further aspect of the flexibility of the system, a display 114 isprovided allowing the display of programmable sound creation components,or musical components termed “words”, governing pitch, tone, scale,effects or synthesizer values as well as other user interfaces orinstruments, aspects of performance and so forth in the form ofmanipulable blocks. These blocks can be manipulated by user selectionfor example, once again, using the interface to select and combinevarious blocks to create phrases providing program commands for theinstrument. Yet further, the components can be stored in memory 112(shown in FIG. 1) and operated under the instructions of the processor110 allowing a modular and flexible controlled approached.

Various functions performed by the system can be performed by agents,aspects of the system can be stored in a data storage structure such asa tree structure including root, and node (branch and leaf) elementswhere check or hash values are assigned to each node as a function ofthe data stored in the structure representing the state of the agent.For example when information relating to one of the agents is stored insuch a tree structure then the check values will be related to thecurrent state component information. If the information is updated—forexample if changed under control of the user then corresponding changesto the data structure can be quickly identified by identifying anychanged check values determining which subtrees of any particular nodehave changed and traversing the tree to establish what check values havebeen changed. In addition to display of an interface allowingconsolation of words to make commands, the data storage structure itselfcan be represented graphically and as a result rapid update andidentification of changes to the system can be identified. Because ofthe manner which data is stored, the state of the system as a whole canbe reviewed and understood in its entirety both by any agent includingan implementation or language agent, and by visual display of the datastructure in a “birds eye” view.

Referring to FIG. 1, it will further be seen that multiple musicalinstruments 116, 118 can be used in conjunction with the system allowingensemble performance and can interact directly with the processor 108 orcan interact via additional/processors/interface modules 120.

Description Of Embodiments

Key Arrangement

FIG. 3 shows a sectional view of a user input or key assembly 125including an optical sensor 140 and a printed circuit board 150 thatforms the basis of the mechanical-human and mechanical-electricalinterface of the musical instrument 100. FIG. 4 is an exploded view ofthe same components. The key assembly also includes a key top 215 thatprovides a user actuatable element and a housing 220 that is located onthe printed circuit board 150. The printed circuit board provideselectrical routing to the components of the optical sensor 140 that arefitted thereto, as will be described below.

The housing 220 has a generally rectangular exterior shape and itprovides rigid mechanical support for the components of the keyassembly. The housing 220 is molded to provide an interior surface withshelves, recesses and lugs as necessary on which the components of thekey assembly can be positively located. The housing 220 is affixed tothe PCB 150 via clips 222 and shown in FIG. 4 or by glue or otherappropriate means. In an embodiment, the clips 222 may be replaced byalignment pegs (not shown) or the housing may rest on top of the PCB.

The PCB 150 has fitted thereon a photo detector 170 and four LightEmitting Diodes (LEDs) 302, 304, 306, 308 that are uniformly spaced in acruciform shape around the photo detector 170. A further feedback LED318 is fitted on the PCB 150 for providing a visual indicator of the keystatus to a user. In an embodiment, two or three or several differentlycolored LEDs are fitted adjacent one another and are used to allowdifferently coloured signals to be generated to denote different levelsor aspects of the key status. One or more bi-colored or tri-colored LEDpackages could be used to achieve the same effect. The optical sensor140 consists of the LEDs 302-308, the photo detector 170, a prismaticsensor light guide 180 and a shutter component 290. The LEDs provide alight source for the optical sensor system. The prismatic sensor lightguide, hereinafter referred to as a “light guide” is located immediatelyabove the LEDs and photo detector as shown in FIG. 3 and provides alight path between the LEDs and the photo detector.

The photo detector 170 collects light emitted from all four LED, oncethe light has been focused through the light guide.

As best seen in FIG. 5 a, the light guide 180 comprises four outerprisms 181, 182, 183, 184 arranged around a central prism 185. The lightguide is manufactured from clear Lexan™ having a refractive index of1.59 or higher. The outer prisms 181-184 are each generally scalenetriangular shape in cross section as seen in FIG. 5 c and comprise anouter sloped, slightly arcuate surface 186 that is arcuate in thevertical plane as shown in FIGS. 5 b and 5 c. An inner surface 187 ofthe outer prism 181 is also arcuate in the vertical and horizontalplanes.

The central prism 185 comprises a generally quadrilateral shaped elementhaving outer walls 188-191 and is situated centrally amongst the outerprisms 181-184 such that a gap is formed between each of the four outerwalls 188, 189, 190, 191 of the central prism and the four inner walls187 of the outer prism. The walls 188-191 are slightly arcuate in thevertical planes in order to help direct the light beam entering thecentral prism onto an interior of the central prism. The interior of thecentral prism comprises four generally triangular shaped walls 192, 193,194, 195 angled downwardly towards the center of the prism. These wallsare very slightly curved in the vertical and horizontal planes.

As best seen in FIG. 5 a and FIG. 5 c the outer prisms 181-184 andcentral prism 185 formed a pathway through which light emitted from oneof the LEDs 302-308 is directed and focused. By way of illustration,FIG. 5 c shows a cross section of outer prism 181 and a portion ofcentral prism 185. An air gap 196 is formed therebetween. Light raysemitted from LED 304 are columnated on entering the outer prism 181through a lower wall 198 thereof. The light rays are refracted as theyentered outer prism 181 towards arcuate outer surface 186. Upon hittingthe outer surface 186 the light rays are reflected internally towardsthe inner surface 187 of the outer prism 181. The light rays exit theprism 181 at surface 187 and pass through the gap 196 and re-entered theprism 185 at surface 188. The curvature of the surfaces of the centralprism 185 and the outer prism 181 are such that the light rays aremaintained as parallel as possible as they exit the outer prism 181 andenter the central prism 185. The light is then refracted upwards towardsthe interior wall 192 of the prism 185 where the light reflects from thewall 192 internally towards a lower wall 199 of the central prism. Uponexiting the prism, refraction of the light focuses it onto the photodetector 170. It will be noted that the specific dimensions andcurvature of the surfaces in the prism elements have been designed tocontain and direct the light emitted from the LEDs and also to provide a“lightgate” in the air gap 196 through which the light beams may pass itis considered that the specific curvature and dimensions of the prismelement are within the remit of the skilled person to design and willdepend upon the dimensions of the key assembly 125 in general.

The key assembly 210 further includes a shutter component 290 that issupported by a spring element 200 that rests in the housing 220. Thespring element 200, seen in FIGS. 6 a-6 d, comprises a generallyrectangular frame 230 that is manufactured from a cold rolled stainlesssteel sheet, the frame 230 including notches 235 cut into the outer edgethereof for mateably receiving corresponding lugs 240 in the interior ofthe housing 220 for location of the spring element 200 in the housing.The spring element 200 further comprises a pair of arms 245, 246 thatextend inwardly from each of the shorter sides 232 a, 232 b of the frame230. The arms 245, 246 are profiled in a U-shape in a direction towardsthe PCB 150 once the spring is assembled into the key assembly. The arms245, 246 are terminated by a bridging member 247 upon which the shuttercomponent 290 is affixed, such that the shutter component is resilientlysuspended and supported on the arms so that it can be pressed down orlowered/tilted or rolled without folding the outer frame 230 and hasfreedom of movement in three axes X, Y and Z shown in FIG. 6 a.

The shutter component 290, best seen in FIGS. 6 a, 6 b and 6 c comprisesa rigid glass reinforced nylon component that is molded over thebridging member 247 of the support members. The shutter component 290 isa four sided component having an upper part that supports the key top215 and transmits force from the key top 215 to the spring 200 and whichfurthermore supports a key seal 250 in position on the key assembly 125,and a lower part is molded beneath the spring 200 and which includesfour side walls or shutter blades 222, 224, 226, 228. Once the key isassembled, the spring 200 rests in the lugs 240 of the housing 220 suchthat the shutter blades 222-228 are supported at a certain height aboveeach gap 196 formed between the outer prisms 181-184 and the centralprism 185 of the prismatic element lightguide 180.

The arms 245, 246 of the spring 200 include a U-bend profile as seen inFIGS. 6 a and 6 b, the U-bend extending in a direction away from the keytop 215. The U-bend profile ensures that whatever possible directionforce is applied to the key top 215, it always results in all fourshutter blades 222-228 moving downwards towards the prismatic element.Should any key actuation result in one side of the shutter movingdownwards and the other side moving upwards, a null key press willresult.

The U-bend profile includes a necked portion 249 best seen in the viewof the flattened profile of the spring 200 in FIG. 6 d and in FIG. 6 a,such that the arm 245, 246 is narrower at the lowest point of the bendthan it is at the level of the frame 230. The necking of the U-bendprofile has several advantages as follows. Firstly, it helps to evenlydistribute the stresses on the spring 200 created where the U-bend isflexed during the key actuation. It also maximises the overall downwardtravel of the key before the yield point of the spring material isreached. The necking 249 also helps to maximize the number of key presscycles that the key 125 can endure before fatigue cracking of the springelement 200 occurs. Fatigue of the spring 200 is furthermore avoided bythe use of corner radii at the junction of each arm 245, 246 and theframe 230 to reduce the stress concentration in these areas.

Referring again now to FIGS. 3 and 4, the key assembly 125 furtherincludes a rubber seal 250 comprising a generally rectangular moldedrubber component that rests on top of the shutter component 290 in thehousing 220. The rubber seal 250 includes a central aperture defining aquadrilateral flange having an elongate bead 255 at a perimeter thereofthat fits closely over a corresponding trench contoured into the upperpart of the shutter component, preventing the ingress of light, liquid,dust and debris in the key assembly. A perimeter of the rubber seal isshaped to fit closely into the housing 220 to seal the outer part of thekey assembly. The seal 250 is manufactured from carbon loaded siliconrubber that also provides an EMC (Electro-magnetic compatability) shieldto help to protect the electronic circuitry in the PCB from noise. Therubber seal 250 also provides mechanical damping of the key movement toreduce unwanted resonance. For example, as seen in FIG. 3, the crosssection of the rubber seal 250 includes a u-shaped portion at eitherside of a key sub 260, described below. The u-section has been includedto increase the mass of rubber in the key over that which may be presentin e.g. a flat seal surface. The increased mass on its own would improvethe resonant characteristics of the key, but in addition, the increasedlength of rubber reduces the spring rate of the seal. Furthermore, theloaded silicon content of the rubber increases the material density,improving the damping characteristic of the rubber and of the key. Thedamping characteristics may be further increased by introducing a smallair hole into the housing wall. For this to work as a means of pneumaticdamping, the rubber seal 250 must seal effectively against the keyhousing.

The key top 215 is located at the uppermost part of the key assembly125, and is mounted on a key sub 260 that locates the key top 215 ontothe shutter component such that movement of the key top 215 resultsdirectly in corresponding movement of the shutter component 290. The keytop 215 is generally rectangular in plan view and comprises a plasticcomponent with a contoured upper surface designed to maximize thesensitivity of the key to playing gestures. The upper surface is thusshaped as a shallow frusto conical shape, with a center portion that isslightly dished downwards to produce five definite sub-surfaces of thekey top for user actuation. In an embodiment, the key is manufacturedfrom wood. The sectional view of the key assembly seen in FIG. 3 showsthree of the surfaces 216, 217 and 218. For example, a user press of oneof the side surfaces will result in a roll or movement of the key top.These side surfaces may also be used to apply lighter sideways pressuresuch as a vibrato movement.

The key sub 260 is affixed to an underside of the key top 215 by glue orother appropriate fixing means. It may alternatively be made integralwith the key top 215. The key sub 260 includes a spring clip at anunderside thereof that is designed to clip into an aperture 195 in theupper portion of the shutter component 290 to rigidly fix the twocomponents together. Clipping the key sub 260 to the shutter component290 also acts to clamp the rubber seal 250 in place between the shuttercomponent and the key top 215. The underside of the long edges of thekeysub 260 are feathered to facilitate the insertion of one or moreblades of a keytop removal tool (not shown) between the underside of thekeysub 260 and the topside of the shutter component 290 to allownon-destructive removal and changing of the keytops 215 for reasons ofwear and/or style.

As shown in FIG. 3, a feedback light guide 340 comprises a clear Lexan™generally cylindrical member having an upper frusto conical portion. Thelight guide fits over the feedback LED 318 and extends upwards towardsthe key top 215 such that once the key is assembled, a user sees onlythe very top portion of the lightguide. The feedback lightguide 340transmits light from the feedback LED 318 through to the top of the key125 to provide visual feedback of the key status to a user. For example,the feedback LED 318 may be configured to emit light only during keyactuation. Where two or more feedback LEDs 318 are provided, each LEDmay be configured to emit light to give feedback to the user of thekeyboard configuration when settings are changed. Another example of useof the LEDs is to present to a player a number of different options inthe natural language interface. In an embodiment, the feedbacklightguide 340 has a roughened top surface that is designed to mix anddiffuse the light output from the LED where a bi-colored or tri-coloredfeedback LED is used, so as to allow the display of all colors presentin the spectrum.

In use of the key assembly, a user presses or otherwise actuates, e.g.rolls, yaws etc the key top 215 to cause movement of the key top 215.The movement is directly transmitted through the key sub 260 to theshutter component 290. The four shutter blades 222-228 of the shuttercomponent each move downwards accordingly to occlude light to the fourlight gates gap 196 as the key is played. Meanwhile, the LEDs 302-308are cycled sequentially to emit light as will be described furtherbelow. The light passes through the outer prism of the prismatic elementand is occluded at least to some extent by the shutter blades movinginto the light gates. The level of light that reaches the photo detector170 is thus representative of not only the end event of a key press, butalso the level and direction of the key press/play and is converted atthe PCB into an electronic signal that can be sent to the processor 108.

As the key is released, the shutter component 290 springs back away fromthe light gate such that no key-actuation event is detected by theoptical sensor 140 until the next time the key is played.

Detection of Key Movement

As described above, the detection system for detecting not only theevent of key actuation but also pressure, roll and yaw values includes asingle photo detector 170 per key together with multiple—in this casefour—LED or other light or radiation emitting elements 302, 304, 306,308 as shown in FIG. 7. In the embodiment shown, the four LED emittersare provided at 90° intervals allowing easy mathematical processing toassess the manner of actuation of the key but it will be appreciatedthat other numbers of LEDs and distributions can be provided. Forexample three LEDs at 120° intervals provide sufficient information toderive all aspects of key actuation. In the embodiment shown, thephotodetector is a photodiode although other photodetectors such as aphototransistor may be used as an alternative.

In addition, a processor which can be any appropriate processor such asa CPU (not shown) is provided and controls, in conjunction with a clock320 both cycling of the LEDs and synchronization of detection at thedetector 170. It will be noted that in operation the multiple detectors170 may be provided each with respective sets of LEDs and controlled bya common clock in conjunction with a single or respective processors anda single or multiple sampling amplifiers.

From the point of view of the control electronics, the keys can bethought of as being grouped together into several ‘strips’ of keys, eachof which share common scanning circuitry. FIG. 7 shows a simplifiedschematic of the key scanning electronics for 2 keys within a strip.FIG. 8 is a representation of the digital control sequence used to scanseveral strips of keys (in this example, 2 strips of 4 keys per strip).Referring to FIG. 8, for, say, the first strip of keys, it will be seenthat the clock generates a clock signal 320 which provides a timing andsynchronization signal for the processor, emitters and detector.

‘Row’ control signals 322, 324, 326, 328 are applied to switches 327(for example FET's—Field Effect Transistors) that supply a power signalto all of the top LEDs in each key of a strip, all of the left LEDs, allof the Right LEDs and all of the bottom LEDs, respectively.

Column control signals 330, 332, 334, 336 are applied to switches 331that connect each of the four LEDs within a single key in the strip toground, such that signal 330 controls the first key of a strip, 332 thesecond key, and so on.

Together the ‘Row’ and ‘Column’ control signals form a matrix whichallow each individual LED in a strip to be switched on separately. Boththe Row and Column signal for each LED must be on for the LED to be lit.For a strip of keys, therefore, a sequence of 16 Row and Column signalcombinations is used to light each individual LED one after the other.Each of the signals 330 to 336 include an on phase that is less than aquarter of the duration of the on phase of the signal 332 to 338 and aretreated sequentially such that, while signal 322 is in the on-state theneach of signals 330, 332, 334, 336 is sequentially the on state.Accordingly, during this time, each of the top LEDs in the first stripof keys are sequentially lit.

The signals 330 to 336 again switch through sequential on states whilesignal 324 is in the on state as a result of which each of the leftLED's in the first strip sequentially switch on and so forth. It will beseen that similarly, each of the LEDs on the second strip aresequentially switched by signals 338, 340, 342, 344.

Furthermore the detectors in a strip are connected to a commonamplifier/filter (not shown). The filter has a slow response (timeconstant) as shown in FIG. 9 selected to ensure that the amplifier justapproaches its maximum output voltage 354 (for the amount of lightcurrently reaching the associated photo detector) during the length oftime that a single LED is lit. This provides optimal filtering fornoise. The amplifier output is sampled by an analog to digital convector(ADC—not shown) common to all keys close to the end of the LED ‘on’period. This is achieved by briefly closing a switch from the amplifierto the ADC, with the control signal 349.

It will be noted that the column signals, for example signals 330 and338, for key 1 in the first and second strips of keys are slightlystaggered. This allows for an improved detection system at the ADC whichcan be further understood with reference to FIG. 9 a. The amplifiersampling period is triggered by control signal 349 which switches on atthe end of control signals 330-336 for each LED, such that the detectoris sampled briefly at the end of each LED illumination period. The otherstrips of keys are scanned in an identical way, but with a small offsetin time. Because the ADC reading only takes part of the total time thata single LED is lit, it is possible to take several readings with theADC during an LED ‘on’ period, requiring just a single ADC. This allowsthe strips of keys to be scanned in parallel, with the ‘row’, ‘column’and crowbar control signals for each strip offset slightly in time. Thisis illustrated in FIG. 8 by showing the column control signals 338-344for the second strip of keys, which are slightly offset in time from thecolumn control signals 330-336 of the first strip of keys.

An alternative scanning approach is also used, where each strip of keyshas its own ADC. In this case all strips can be scanned together inparallel, and therefore the sets of control signals associated with eachstrip of keys do not need to be staggered in time.

In order to allow for optimal noise filtering of the amplifiedphotodetector signal, once the signal has been sampled by the ADC it is“crowbarred” or grounded using the control signal 347 shown in FIG. 9 aas can be seen at 356 so as to discharge the device and ensure that onthe subsequent detection cycle the value starts at zero.

‘Crowbarring’ is a normal electronic technique that will be known to theskilled reader, used to clamp a signal to a fixed value quickly. Howeverits use according to the present aspect provides a surprising effect. Aproblem with the amplification of the signals from the photodetectors isthe tradeoff between noise and amplifier settling time. A rapid settlingtime implies a wider bandwidth amplifier which produces a lot morenoise. However the present approach allows an amplifier configured tohave a much lower bandwidth. At the point its value is sampled it hasnot settled fully to its final output. However as it is possible todetermine the sampling time very accurately timing accuracy is tradedfor absolute accuracy in the amplifiers, giving a consistent result. Thecrowbar is needed to return the amplifier to exactly the same initialstate each time in order to prevent the value from one samplingiteration affecting the next, thereby avoiding crosstalk betweensensors.

In addition it will be noted that the specific amount of depression ofthe shutter associated with each LED can be computed from theinformation provided. In particular, through one detection cycle wherebyall four LEDs around the detector are illuminated in sequence, thecorresponding voltage value from the detector is sampled and stored.This voltage value, is of course, representative of the amount of lightreaching the detector from each of the LEDs which in turn is a measureof the level of depression of the key in that direction and thecorresponding interruption of the light signal by the shutter. It willfurther be noted that the system can include any number of keys in anyconfiguration together with one or more alternative function key such asa control key, operating in the same manner as discussed above.

In one embodiment, during a calibration phase, the specific relationshipbetween the amount of depression of the shutter and the correspondingsignal at the detector can be correlated and stored. For example the keycan be depressed by a series of increasing, known amounts and thecorresponding voltage signal for each amount identified. Then a lookuptable can be created correlating each depression value with thecorresponding voltage value. In operation, therefore when a specificvoltage value is detected at the detector, the corresponding depressionvalue can be derived from the lookup table for example by linearinterpolation or a more sophisticated interpolation function dependanton the calibration techniques adopted. Values such as roll and yaw canbe obtained by comparison of sensed values of depression of opposingedges of the key.

The raw sensor data consists of four 12 bit values per key, one perleft, upper, right and lower edges of the key top. Each sensor isscanned at a rate of 2 kHz. For example in one embodiment thecalibration process generates a set of readings of key sensor values andthe corresponding force exerted on the key. From this data, for eachedge, a number of points are generated corresponding to equal divisionsof sensor reading between the minimum and maximum values (at the ends ofthe key's excursion) and the corresponding key forces as measured on acalibration jig. These forces are normalized and placed in a datastructure which performs a piecewise linear interpolation to convert araw sensor reading into a normalized force value.

Changes to the sensor zero value can affect the sensitivity of thesensor to human touch as it is used. If the user presses down with apositive force on a force sensor for some seconds the materials in thesensor distort into a new shape. If a force sensor system issufficiently sensitive then it will suffer from the effects ofhysteresis in the physical materials from which it is made. When thesensor is then released, the distortions may mean that the sensor eithercontinues to indicate that it is still experiencing the application ofpositive force or, if the system has distorted in the oppositedirection, the application of negative force. Some of these distortionswill last for a finite time and will have a number of effects on theaccuracy of the sensor. Slow changes in the sensor caused by thermalvariations may also cause the sensor to become inaccurate.

The system that determines whether or not a key has been pressed (whichmight be, for example an electronic or software system) now has one oftwo outcomes. In the first case the system will still seem to be abovethe sensor zero value thus indicating that force is still being exerted,and it will remain this way until all the materials in the sensor haverelaxed back to their original shape, which might take some time. In thecase of a musical instrument this can result in a note becoming ‘stuckon’. In the second case the value from the force sensor is now below thezero force value by some amount and will now require the application ofincreased force in order to trigger the minimum force value. In themusical case this results in a key that has become less sensitive as aresult of being played. In addition to the hysteresis effects,components of the force sensor may sometimes also exhibit resonantbehavior during rapid changes in the applied force, particularly whenthe force drops from a large value to zero in a short period of time.Both of these results are undesirable and the present inventionaddresses these problems. In order to be able to determine if a forcehas been applied to the sensor the effective zero value is adjustedduring the time that it is pressed to accommodate the changing materialshape, and when it is released from the applied force it is modified toreflect the relaxation. It is also desirable that the process thatdetermines the zero force value may account for the presence ofoscillations.

In the particular case of a human touch interface, and in the even moreparticular case of one used for the production of musical sounds, theinvention recognizes the possibility of taking advantage of some humanbehavior. Whilst one often plays notes that start very quietly and thencrescendo loudly, it is extremely rare that one starts with a loud noteand very slowly eases it all the way to off. Normal behavior when slowlyapplying diminuendo to a note is to apply it for a while until the noteis fairly quiet and then just ease all the way off quickly. Using thisfact enables the application of a simpler method for maintaining theforce sensor zero value that does not require full modeling of thephysical system and can be realized in simple hardware or software, asthe cases of crescendo or diminuendo are particularly accommodated.

The manner in which this is achieved can be understood from FIGS. 9 band 9 c. Referring to FIG. 9 b a threshold filter 930 generates anoutput value that represents the force applied to the key (CENTERFORCE). To do this it uses a Low Pass Filter (LPF) with a long TimeConstant (TC) to generate a baseline ‘zero’ level for a key when it isnot being pressedbased on the raw ADC values 931. The Low Pass filterthus compensates for long term drift in the baseline ‘off’ reading fromthe key, due to (for example) thermal effects in the electronics. Theoutput 932 of the threshold filter (CENTER FORCE) is an averaged readingfor the key and will represent the force applied to the key plus anymechanical or electrical noise (thermal noise, mechanical cross-talkfrom adjacent keys etc).

The threshold LPF is turned off (retains its previous value) when thekey is active, and also for a short period such as 5 seconds after thekey has stopped being active to prevent the zero level from changingwhilst the key is being played or is mechanically settling after havingbeen played. A shorter Time Constant is used in the LPF for the first 20seconds after the instrument is powered up. This is because there is aninitial change in ADC readings for the zero level whilst the electronicswarms up. This allows the filter to react to this change more swiftly.

An Activation Filter 934 takes the CENTER FORCE value from the ThresholdFilter (i.e. the sensor level representing the force applied to the key)and uses this to generate an ACTIVATION THRESHOLD value 936. This is thethreshold used to determine whether or not the activation state of thekey has changed. In order to generate the ACTIVATION THRESHOLD anadditional fixed THRESHOLD OFFSET value (935) is added to the CENTERFORCE (generating the OFFSET FORCE). This is used to compensate forelectrical and mechanical noise in the system by effectively increasingthe ACTIVATION THRESHOLD by a value that represents the maximum expectednoise.

As described in more detail below, in addition, a de-sensitization key(or ‘dummy’ key) is provided mechanically identical to a standardkeyboard key but not intended to be used as a performance interface, butmerely to mirror the mechanical properties and hence provide a controlvalue allowing detection and desensitization, for example, tosympathetic resonance or mechanical knocks. It will be noted that thesecan be manifested as both positive (key depress) forces, as encounteredby a standard key, but also as a negative force for example fromresonances.

A de-sensitisation filter 970 has as inputs the raw sensor ADC values971 and uses a first stage key filter that is identical to the keyfilter of a normal key, including the storage of the active/inactivestate. The active state is only used internally, however, to control thefilter characteristics of the Low Pass Filters in the key filter. Valuesfrom the ‘normal’ Threshold Filter are then used to generate an absoluteforce value for the ‘dummy’ key. This produces an output when the key isgoing up from the zero level filtered value as well as down. Thisabsolute force reading provides a good indication of the magnitude ofany un-wanted transients on the other keys due to external factors suchas the instrument being knocked, or sympathetic resonance.

The absolute force value for the ‘dummy’ key is scaled using a usersettable parameter 972 and the scaled force is then passed through a LowPass Filter with a reasonably short time constant. The output of thisLPF is a DESEN FORCE value 973 which can be used to de-sensitize theactivation of the main keyboard keys as discussed below.

The activation filter has 3 different modes of operation and theACTIVATION THRESHOLD value is generated from the OFFSET FORCE valuedependent on the mode, taking into account, inter alia, whether the useris inputting crescendo or diminuendo. This is implemented by the statemachine 938 to identify the key activation state 939 by comparing theACTIVATION THRESHOLD from the Activation Filter with the CENTER FORCEfrom the Threshold Filter, and change the activation state accordingly,and can be further understood from the state diagram of FIG. 9 c:

In the first, “active” mode, if at step 950 the key is active then atstep 957 the last value of the ACTIVATION THRESHOLD prior to the keygoing active is stored, a fraction of the OFFSET FORCE is mixed with thestored value of the ACTIVATION THRESHOLD. This is passed through a LowPass Filter with a relatively short time constant to generate the newACTIVATION THRESHOLD. This allows the ACTIVATION THRESHOLD to trackquick or slow downward motion, i.e., increasing applied force and sensorvalue (normal or crescendo) and relatively slow upward motion(diminuendo) on the key without a change in activation state, as in thecase of increasing force it will be greater than the ACTIVATIONThreshold as it is recomputed in step 951, and a slowly decreasing forcewill also be above the recomputed threshold. However the system reactsquickly to swift upward motion (decrease of force and sensor value), asthe sensed value falls below the ACTIVATION THRESHOLD value before ithas time to decrement, hence de-activating the key at step 952.

In a second, “charging” mode, at step 950 the Key is in-active and goingup from mechanical settling. In this case the OFFSET FORCE is passedthrough an LPF with a short time constant to generate the ACTIVATIONTHRESHOLD. This allows the threshold to quickly track upwards settlingof the spring after the key has been released. The value is furthercorrected with the desensitization force to filter out externallyinduced transients and at step 954, if the sensed CENTER FORCE is lessthan the desensitized ACTIVATION THRESHOLD, then the key state remainsinactive. Otherwise at step 956 the step is set to ACTIVE.

In a third, “discharging” mode, the Key is in-active and not going up,i.e., has settled. In this case the OFFSET FORCE is passed through anLPF with a relatively long time constant to generate the ACTIVATIONTHRESHOLD and step 954 can be repeated. This helps prevent the thresholdfrom reacting to noise once the spring has settled.

The charging time constants are set by the material characteristics anddesign of the force sensor system, for example taking into account theresonant characteristics of the force sensor system to avoid inadvertenttriggering of the sensor by oscillations caused by force release. Thenoise threshold is determined by both the noise inherent in the sensorsystem and by the degree of sensitivity to small forces desired in thetotal system. It is also determined by the speed at which thermaleffects may make the sensor value drift as it interacts with the timeconstant described below for the thermal filter. If it is notsufficiently large then changes in value caused by temperaturedifferences may cause false triggering before the thermal filter haseffect.

The threshold filter may be for example a one pole IIR low pass with along time constant of the order of one minute.

The activation filter may comprise a one pole IIR low pass filter with atime constant of the order of 0.1 seconds, when the key is active; ofthe order of 0.05 seconds, when the key is inactive and the input valueis less than the previous output; and of the order of 5 seconds, whenthe key is inactive and the input is above the previous output. When thekey is active, the input to the activation filter is scaled by a factorSCALE, which is less than 1, typically 0.2, so that the activationfilter charges up to a fraction of the input value.

In one embodiment the values are:

Activation Filter Active time constant: 100 ms

Activation Filter Active coefficient 0.2

Activation Filter Discharge time constant: 50 ms

Activation Filter Charging time constant 5 sec

Threshold Filter time constant: 50 s

An embodiment of the musical instrument is shown in FIG. 11 a. Themusical instrument 100 will, in use, be subjected to physical shocksthrough general usage, wear and tear and will also be exposed to soundsfrom outside of the instrument. This can cause a problem in that thekeys 125 of the musical instrument 100 could be shocked or the springcomponent 200 of the keys 125 could be excited by resonant frequenciesfrom external noise. In a live situation for example, a positivefeedback loop might form between the mechanical key resonance and anexternal sound system. This could adversely affect the sensitivity ofthe optical sensor 140 and the performance of the keys 125. In order toaddress this problem a dummy key 950 is mounted inside the musicalinstrument 100, for example it may be mounted on the PCB 150 but notexposed to the user of the instrument. The dummy key 950 cannot beplayed by a user, but is comprised of a complete key assembly 125,including optical sensor 140, key sub 260, key top 215, key seal 250 andhousing 220. The dummy key 950 is electronically connected via the PCB150 to the processor 106 where activation of the key 950 is detected inthe same manner as with the playable keys 125.

If activation of the dummy key 950 is detected, it can be concluded thatthe musical instrument 100 has been subjected to a physical shock or toan external noise causing a resonant frequency at the keyspring 200.Whatever the cause of the activation, it will trigger the processor 108to de-sensitize the detection of signals received from the playable keys125 by an amount substantially the same as, or a little more than, thesignal detected at the dummy key 950. Thus the user can continue playingthe keys 125 even if the instrument 100 has been knocked by accident orhas been adversely affected by external noise.

Alternatively, the key may be placed on the outside of the instrumentwhere it may also serve the function of a ‘mute’ key that desensitizesthe whole instrument to the degree with which it is pressed. A gesturerecognition on that key (a deflection sideways for example, instead ofstraight down) latches the muting effect on or off.

It will also be noted that the form of the dummy “key” can match anyphysical form of the user input. For example in feedback suppression ofacoustic and electric guitars, by building in dummy strings for example,the output value of these can be used to desensitize the instrument (byturning down its volume). In any case, the use of a sensor attached tothe instrument that shares the same mechanical characteristics (perhapsincluding the resonant frequencies, etc.) of the active sensors used foractual data input, allows the output (the filtered output in fact) to beused to desensitize the other sensors.

It will be noted that the functionality of picking up mechanical shocksmay alternatively be replicated by the use of an accelerometer in theinstrument which can also be used to provide performance data.

Instrument Frame

In an embodiment, the PCB 150 has an array of key assemblies 125assembled thereon, for example a full scale musical instrument mayinclude an array of 5×24 keys whilst a smaller version may include only3×6 keys. At least the full scale instrument is designed to have anergonomic and aesthetic curved frontage, as seen for example in FIG. 11a. A smaller version may have a more conventional flat-fronted keyboard.In either case, the key assemblies 125 may be located in very closeproximity to each other on the PCB 150. The sensitivity of the opticalsensor and key assembly is particularly high and may detect movement ofthe key with a sensitivity of as little as 0.9 micrometers. It istherefore very important that the keys are not affected by movement ofadjacent keys. For the flat-fronted keyboard, minimization of this“cross-talk” between adjacent keys is achieved by using a rigid castframe (not shown) that fits over the keys and clips into a rigid base ofthe instrument to provide a uniform clamping action, over the wholekeyboard, of the keys between the base and the frame. This prevents anycross-talk between keys that may otherwise occur via the rubber seal 250of the keys and also provides the front cover for the instrument.

However, for the curved-fronted version of the instrument, the frame 400is laminated as this is a more cost viable solution than other possibleforms of manufacture such as CNC machining. The laminate structureprovides the rigidity required to uniformly clamp the rubber seal 250 ofeach key between the key assembly and the frame as described above suchthat the clamping force is evenly distributed. FIG. 17 b shows aperspective view of a portion of an example laminated frame, whilst FIG.17 a shows an exploded view of the frame layers. The frame comprises alaminate structure manufactured from aluminium or steel. In the exampleshown, the laminate structure comprises four thin chemically milledmetal under-cages that are glued or otherwise stuck together to form arigid, curved frame. The outer cage 401 is attached to and pulledtowards the instrument base using springs or other fixings (not shown).The laminate structure produces a strong, rigid frame even though itcomprises apertures to accommodate each key. The frame may also providea front cover for the musical instrument through which substantiallyonly the key top 215 and the tip of the feedback LED 340 protrudes. Therigidity of the frame minimizes the risk of the playing of one keycausing detection of movement in the key assembly of the adjacent key.

Breath Controller

In addition to the keys 125, a musician may utilize a further input,namely a breath controller 130. An embodiment of the breath controller130 is shown in FIGS. 10 a, b and c. The breath controller 130 consistsof a breath pipe 702 that has a proximal end 701 that is in fluidcommunication with a mouthpiece 704, and a distal end in fluidcommunication with pressure transducer 706. The mouthpiece 704 comprisesa valve body 708 which consists of two halves 708 a, 708 b as seen inFIG. 10 b, and an interchangeable portion 710 through which a user mayblow, suck or hum to create static pressure inside the valve body 708.The interchangeable portion defines an internal cavity 709 that isshaped to conform with an exterior surface of the valve body 708 suchthat the interchangeable portion 710 snap fits onto the valve bodyexterior surface and is removable to allow different players to sharethe musical instrument 100 and to use the breath controller 130 with anincreased level of hygiene.

The valve body 708 comprises a molded rigid plastic component molded inthe two halves 708 a,708 b but are substantially symmetric to each otherabout the plane at which they are joined together as shown in FIG. 10 b.The valve body includes an airflow inlet 712 and an airflow outlet 714defining a valve cavity 716. When the interchangeable portions 710 isattached to the valve body 708, the internal cavity 709 and the valvecavity 716 together define an airflow cavity 720 through which air flowsduring use of the breath controller. The pressure chamber 718 is offsetfrom and is separated from the airflow cavity 720 to a large extent bydividing wall 719 such that air does not flow through the pressurechamber 718 during use of the breath controller. This arrangementreduces the build up of spittle and condensation in the pressure chamber718 as a user blows, sucks or hums into the interchangeable portion 710.However, the pressure chamber 718 is in fluid communication with theairflow cavity 720 such that as a user creates airflow through thecavity, a positive or negative static pressure is generated in thepressure chamber 718 depending on whether the user has blowed, sucked orhummed into the mouthpiece.

The valve body 708 further comprises an aperture 725 defining anentrance into an interior channel 730 that is in fluid communicationwith the pressure chamber 718 at an end of the valve body opposite tothe interchangeable portion 710. The proximal end 701 of the breath pipe702 is received in the interior channel 730 for attachment to themouthpiece 704. Two pipe seal O-rings 732,734 are fitted inside thechannel 730 around the pipe 702 to provide an airtight seal of the pipeinside the valve body 708. This ensures that any static pressuregenerated inside the pressure chamber 718 is sealed from the environmentand is also generated in the breath pipe 702. Similarly, O-ring seals732,734 are provided at the airflow inlet 712 at an exterior surface ofthe valve body 708 in order to seal the interchangeable portion 710 onthe valve body. A gasket 739 is fitted between the two halves 708 a,708b of the valve body in order to provide airtight seal between the twohalves of the valve body. The mouthpiece 704 is thus airtight other thanat the airflow inlet 712 and airflow exit 714.

A variable bypass valve 750 is located at the airflow exit 714 in orderfor the user to be able to adjust a rate of airflow passing through theairflow cavity 720 and hence to adjust the static pressure generated inthe pressure chamber 718. The variable bypass valve 750 comprises anadjustable thumb screw 740 located on the exterior of the mouthpiece,the shank of which is passed through an aperture in the valve body 708close to the air exit 714, the aperture being perpendicular to theairflow exit 714 such that the further the screw is pushed or turnedinto the valve body, the more it closes off the bypass valve 750,reducing the cross-sectional area of the airflow path in/out of theairflow cavity 720, altering the static pressure generated in thepressure chamber 718. The user may thus adjust the thumb screw to adjustthe rate of airflow through the bypass valve 750. In another embodiment,the variation of airflow through the bypass valve could be achieved viaanother appropriate type of valve such as a ball valve or a gate valve.

The valve body halves 708 a,708 b are joined together with a fixingscrew 760 in this embodiment although any appropriate fixing means maybe used. The O-ring seals 736,738 allow the mouthpiece 704 to be rotatedwith respect to the breath pipe 702 to provide an optimal playing anglefor the user.

In use of the breath controller, a user may blow, suck or hum into theinterchangeable portion 710 of the mouthpiece 704. This action createsan airflow through the airflow cavity 720. The rate of airflow throughthe bypass valve 750 can be controlled by adjusting the thumb screw 740to increase or decrease a rate of airflow through the valve. The airflowgenerates a static pressure in the pressure chamber 718 that isproportional to the flow rate through the valve and the airflow cavity.The static pressure is sensed at the pressure transducer 706 at thedistal end of the breath pipe 702. The transducer, present in themusical instrument 100, converts the sensed pressure into an electricalsignal for processing by the electrical-processor interface 106 of themusical instrument 100. Spittle and condensation generated during use ofthe breath controller are vented through the airflow exit 714.

Instrument Strap and Strap Attachment

Referring to FIG. 11 a, a strap 920 is provided in order that theinstrument 100 can be hung from a user's body. The strap is attached tothe instrument via an attachment means 30 that allows the instrument topivot around its exact center of mass, affording the musician themaximum number of possible playing positions and ease of change from oneplaying position to the next.

The instrument sits best on the user in an off center position and atapproximately hip level. In order that the instrument hangs in thisposition, the strap 920 consists of a main strap 921 and a substrap 922,seen in FIGS. 11 b, 11 c and 11 d. With reference to FIGS. 11 b, 11 cand 11 d, the main strap 921 hangs over the shoulder of the user in anormal manner to the front of the user, terminating at the attachmentmeans 30 to which it is fastened by rivets or other appropriatefastening means. FIG. 11 c shows a rear view of a user wearing the strap920. The main strap 921 is worn such that the strap curves down andacross the back of the user and around the user's waist, from where ithangs at the user's front and is clipped to the attachment means 30. Thesubstrap 922 is shown in FIG. 11 c to depend from the main strap 921along the upper curved part of the main strap at the rear of the user.The substrap 922 is clipped to the attachment means as seen in FIG. 11 band FIG. 11 d. The substrap 922 therefore locates the instrument 100 inan off center position on the user and prevents it from reverting to acentral position. Both the main strap 921 and the substrap 922 areadjustable so as to allow the user to reposition the strap relative tohis or her body. The main strap, for example, can be adjusted at a frontbuckle 923, whilst the substrap 922 can be adjusted at the point ofattachment to the main strap. The main strap 921 includes a rigidportion 924 that extends from the point of connection with the backplate 16 to approximately just above the breast of a user. The rigidelement 924 produces a lever force that is necessary to support theinstrument on the attachment means.

The attachment means 30 slots into a recessed pocket 20 in a base 405 ofthe body of the musical instrument 100 in order that the instrument ishung from its center of mass.

The attachment means comprises a balljoint device that allows the userto adjust the friction applied to the pivot from very little(substantially free movement) to a fully locked-off state in which theballjoint cannot be rotated. In the locked-off state, the user can playthe keys of the musical instrument with both hands, without having tocounterbalance the instrument with the hand that would normally supportthe instrument, maximizing the flexibility with which a user can createsound using the instrument.

The balljoint device is described as follows with reference to FIGS. 11e, 11 f and 11 g. The balljoint comprises a pivotable ball 5 that isjust seen in FIG. 11 e and is seen in section in FIG. 11 f. Theballjoint 5 is in operable contact with a friction pin 11, via which auser may adjust an amount of locking force on the balljoint during use.

The balljoint ball 5 is operably housed inside a generally cylindricalretaining socket 6, relative to which the ball is pivotable. At a distalend thereof, the retaining socket 6 includes an aperture into which theproximal end of the friction pin 11 is retained. The socket also retainsa proximal end of a hollow stem 8 which extends between the balljointand a strap attachment portion 16, 17 of the attachment means 30. Thestrap attachment portion 16, 17 of the attachment means is, in use ofthe instrument, attached to the strap 920 and the stem 8 thus provides ashort distance between the player and the musical instrument to allowmaximum uninterrupted travel for the user's hands whilst playing theinstrument.

Running inside the hollow stem 8 is a friction pin 11, a proximal end 11a of which is arranged to come into contact with the ball 5 when forceis applied to the distal end 11 b of the pin. The distal end of thehollow stem 8 is threaded and also houses an adjustment pin 11 c, an endof which is threadably received in the threaded portion of the stem 8.An opposite end of the adjustment pin 11 c is affixed to the strapbracket 10. A force may be applied to the friction pin 11 a via anadjustment screw 12 that can be turned into the threaded portion of thehollow stem 8 at an angle perpendicular to the longitudinal axis of thefriction pin 11. Turning the adjustment screw causes the rotation of theadjustment pin 11 c either toward or away from the friction pin 11 a,increasing or decreasing the force applied on the ball joint 5. As forceis increasingly applied to the friction pin 11, the friction acting onthe ball 5 increases, eventually to the point where the ball 5 haseffectively been locked-off and will no longer rotate.

The ball joint 5 is shown in FIG. 11 f to have two conical orfrusto-conical cutout portions 13 a, 13 b at directly opposite endsthereof. The cutout portions extend to an angle of approximately 20degrees from a central axis thereof and meet at the center of theballjoint such that the cutouts extend the entire way through the balljoint. A gimble pin or locating pin 14 extends through an upper surfaceof the retaining socket 6 as seen in FIG. 11 f, through thefrusto-conical cutouts 13 a, 13 b and through a lower surface of thesocket 6 such that it is held in place in the retaining socket 6 but isonly very loosely in contact with the interior of the ball joint at thenarrowest point of the cutouts. Furthermore, the retaining socket 6comprises a cutout portion 15 in an upper portion thereof as shown inFIG. 11 e. The gimble pin 14 is aligned perpendicularly with the cutout15. This arrangement of the conical portions 13 a, 13 b, cutout portion15 and gimble pin 14 allows the balljoint 5 to rotate plus or minus 20degrees within the socket 6 in any direction. At the extremities of thisrange, the gimble pin 14 contacts the conical cutouts 13 a, 13 b andprevents further movement. However, the cutout 15 in the socket memberallows the ball joint to be rotated up to 90 degrees in the direction ofthe cutout before the balljoint impinges on the socket 6. The gimble pin14 thus functions as a locating means for the ball joint such that theposition of the cutout 15 does not change with respect to the balljoint,ensuring that the balljoint and socket do not rotate relative to thebody of the instrument in a manner that could damage the device orinconvenience the user.

The balljoint 5 is retained in the retaining socket 6 by a circlip 23seen in FIG. 11 f. The circlip locates into an internal notch 24 in aninterior surface of the socket to prevent the balljoint 5 from movinglaterally out of the socket. As force is applied to the friction pin 11in the direction of the balljoint, it is the friction generated betweenthe balljoint 5 and the circlip 23 that creates the locking of theballjoint in position.

The balljoint is attached at a distal end thereof (opposite the frictionpin 11) to an instrument mounting device 25 using screws or otherappropriate fixing means. Referring to FIGS. 11 g, 11 h and 11 i, theinstrument mounting device 25 consists of an outer shoe 26 which in thepresent embodiment comprises a folded metal plate having a generallytrapezoidal planform so as to form a tapered sleeve having a narrowingcross section from an entrance thereof to an exit thereof. Inside theshoe 26 is received an inner foot 27 that also comprises a folded metalplate fitting snugly inside the shoe 26. The foot 27 functions as afixing mount for the balljoint 5 and also for a latch mechanism that isreceived inside the foot. The latch mechanism comprises a folded metalspring 29, a handle 30 and a latch 31. The folded metal spring 29, seenin FIG. 11 h, is generally trapezoidal in planform so as to fit snuglyinside the foot 27. The spring consists of a rear flat portion thatincludes an upper mounting portion 32 into which rivet holes 32 a and 32b are formed. Below the upper mounting portion are two arms 34 a and 34b that taper inwards towards a longitudinal axis of symmetry of thespring. The arms are folded forwards on themselves to create a U-shapedspring hinge at a lower end of the spring 29, the arms extending backtowards the upper mounting portion 32. Each arm terminates in aprotuberance 37 that is mateably receivable in a corresponding pair oflocating notches 36 in the foot 27 (see FIG. 11 g).

The latch 31 comprises a ‘Chubb’™ type latch having an attachmentsurface that includes a pair of rivets 39, and a cam surface 38 thatachieves the latching. The rivets 39 are used to fasten the latch 31 tothe spring 29 via the holes 32 b in the upper mounting portion of thespring 29. The handle 30, consisting of a plastics or metal element thatis contoured for ease of user grip, is attached into the holes 32 a inthe spring 29 using rivets.

The spring 29 and latch 31 is then slotted into the foot 27 until theprotuberances 37 on the spring 29 locate within the notches 36 to fastenthe spring into place. During this action, the inner surface of the foot27 rides along the cam surface 38 until the cam surface protrudesthrough an aperture 41 in the foot 27, clicking the latch into place andlocating the spring in the foot. The arms of the spring expand insidethe foot to wedge the spring in place.

As seen in FIG. 11 h, the instrument attachment device further includesa mounting bracket 1 that may be affixed into the pocket 404 in the base405 of the musical instrument 100 using standard machine screws 2 orother appropriate fixing means. Once the latch and spring are located inposition in the foot, a user must pull on the handle 30 to physicallypull the latch out of the aperture 41 to remove the instrumentattachment device from the bracket 1 in order to separate the strap andattachment means from the instrument 100.

The pocket 20 is conductive and is grounded to the rest of theinstrument 100 in order to complete the electromagnetic compatibilityshielding of the instrument. The bracket 1 and the pocket 20 arerecessed into the base 405 such that the ball 5 can be located at theinstrument center of mass.

A distal end of the adjustment pin 11 c is threadably screwed into aT-section 17 that is clamped between inner and outer sections of a strapmounting plate 16. The strap mounting plate 16 is in turn attached viarivets or other appropriate fixing means to a rigid section 21 of themain strap 921. The main strap 921 passes over a shoulder of the user aspreviously described and is attached at its other end to strap mountingplate 16 with a sprung clip 23 such that the instrument is firmly andcomfortably hung from a users body. The strap can be easily unhung bydetaching the clip 23 from the strap mounting plate 16.

There will be occasions when a user wishes to play the sound-creationinstrument whilst sitting down, in the manner of a large stringinstrument such as a cello. In this case, the strap and instrumentattachment means are both unnecessary and undesirable. These componentscan be easily separated from the instrument by unclipping the strap fromthe backing plate 16. To enable the user to position the instrument in acomfortable playing position, an adjustable and retractable spike 77 isincluded at a lower end of the instrument 100 as shown schematically inFIG. 11 a. The spike is retractable to a complete or significant extentinside the instrument 100 and can be telescopically adjusted in lengthand locked at the desired length using a locking screw 77 a that engagesthe spike and locks it in place using a friction force.

It will be understood by the skilled person that the balljoint andadjustment pin mechanism illustrated in the present embodiment is oneexample of the type of mechanism that can be used in order to adjust andlock out the positioning of and amount of friction applied to theballjoint, that to locate the orientation of the balljoint with respectto the socket and that other mechanisms may be employed withoutdeparting from the general principles disclosed herein.

Natural Language Interface

Referring to FIG. 12 the graphical user interface 400 can be seen inmore detail. The GUI displays sound creation components or wordsaccording to a natural language whose syntax can be further understoodas follows:

The system is configured and controlled by a natural language interface,where “sentences” or “phrases” are formed in this language composed ofnouns and verbs and may be formed to set data values, initiate actions,schedule future events and so on.

The sentences act as commands which can configure or program any aspectof sound or music creation such as the sound of an instrument, musicalaspects such as the notes, key or mood corresponding to a set of musicalkeys, control of other instruments in a common performance or otheraspect of the performance such as lighting.

Each sentence may be ended by either a verb or a specific sentenceterminator, and the sentences may be typed in text from a computerkeyboard although more preferably sentences are normally spoken byplaying musical notes, which a language interpreter program decodes.Each word in the language is thus represented by a note or series ofnotes and can be delimited by either a louder note (a note above acertain changeable threshold) or by a specific word terminator note. Thelanguage interpreter program comprises a language “agent”, one of aseries of agents which can be supported by the system providing certaindefined functions.

Notes used to form the words may be on any scale but the major diatonicscale (of 8 notes) is the most useful as this means that the languagemay be easily communicated orally (by singing or playing words toanother musician) or visually (by using the traditional (doh-re-me) handsignals). It may also be communicated by means of the staff or a seriesof numbers. This provides both control of the sounds available to themusical instrument, control of other performance parameters such aslighting or interaction with other instruments and allocation ofcontrols and sound outputs to the specific keys or other user inputs ofthe instrument.

In an illustrative embodiment shown in FIG. 12, for example, it will beseen that the sound components or words include a “set” component 402, a“scale” component 404, a “pentatonic” component 406, a “to” component408 and a “major” component 410. Each component as shown in FIG. 13 isrepresented by a musical stave 430 and a natural language descriptor432. The stave 430 can carry musical notation representing a unique setof notes, corresponding to the command and the descriptor comprises anatural language descriptor 432. For example the stave in FIG. 13carries the crotchet/quarter notes B, C, the corresponding number 5, 6and represents the descriptor “start”. Activation of the notes orselection of the corresponding numbers allows incorporation of the“start” word into a natural language phrase which is then read by thelanguage phrase which is then read by the language agent to start afunction such as a metronome.

The components or words can be combined to form a phrase acting as acommand which is then interpreted by the language agent as discussedbelow which will then cause one or more agents to carry out actions. Aphrase like “all oscillator volume up” will cause all the oscillatorsthat are relevant in the language agent's current context to get louder.In this case, each oscillator will be controlled by a respectiveadditional agent which will store control values (such as volume) whichvalues are rewritten by the phrase as interpreted and implemented by thelanguage agent. Alternatively a separate “volume” agent may be providedto which one or more instrument agents link to provide the relevantcontrol values.

On the other hand the phrase could refer to a specific oscillator by itsdescriptive information, e.g. “the sawtooth oscillator”. The words candefine, therefore, a musical action (such as pentatonic, major or scale)an operator action such as “to” or a command component such as “set”.Upon selection and combination of the components by a user, thecorresponding operation as interpreting the language agent can takeplace. For example where the user wishes to change the scale played bythe instrument then the words can be selected and combined in the order“scale”, “to”, “pentatonic”, “major”, “set”. As a result the languageagent will interpret the corresponding commands to set the scaleperformed by the instrument to pentatonic major.

In addition, where multiple users or instruments are involved, eachparticipant can be assigned, or assign themselves, a unique identifierin the form of a sequence of notes, which can be termed a “name”. Thisapproach can be extended to any other elements of the performance orcollection of elements which may require rapid identification during aperformance allowing disambiguation of participants or elements from theperformance itself.

It will be seen that the manner in which the words or components areselected can be via a standard mouse over and click operation.Alternatively in a preferred approach the components can be selected byplaying the notes on the stave 430 using the instrument as a selectiveinterface. Alternatively again each note on the stave can be assigned anumerical value (as shown in FIG. 13) corresponding to a numerical valueassociated with the keys allowing the interface to select a component byselecting the corresponding numerical code (for example 5,6).

It can further be seen that the components 402, 404 can comprise anyappropriate sound creation component such as setting pitch, tone, scale,effects, synthesizer or can relate to other performance creationcomponents such as lighting control or indeed can allow control of otherinstruments or interfaces associated with the system for example otherinstruments performing in an ensemble as well as language agents foranalyzing the descriptive data. Yet further because of the use ofnatural language identifiers 432 and a natural language syntax developedusing the operators and commands, an easily usable interface isprovided.

In one form of the syntax verbs can be recognized as phrase delimitersbut numerous other possibilities exist including louder/harder keyactuation.

The use of a natural language, communicated as notes sequences encodingwords in a musical scale and used to configure and control a musicalsystem and the concept of representing functions as agents which can becontrolled and linked as appropriate hence allows flexibility andscaleability on an unprecedented scale.

The manner in which the system operates can be further understood withreference to FIG. 14 a. Each agent has a set of operational parametersand values represented by a tree structure which includes a root element500, and nodes having a type value and comprising branch elements 502,each defining a key subtree 504, and leaf elements 506. Each leafelement has associated therewith a data value. For example in the caseof the scale agent or component 404 shown in FIG. 12, leaf 508 maycomprise the scale value (the root note of the scale A, B . . . G) theelement 510 may include the mode (diatonic, chromatic, pentatonic etc)leaf element 512 may comprise as a data value the mood of the scale(minor, major) and other leaf elements can include other data valuesrelevant to the scale.

Reverting to the natural language instruction set discussed above withreference to FIG. 12 it will be seen that a performance will principallyinvolve involving remote procedures in the agent. However, any actionwhich changes the setup is reflected by changes in the state tree. Forexample, the language agent upon implementation of the command “scale topentatonic major set” will rewrite the values within the data structureto reflect the changed values as appropriate. For example leaf element508 relating to the key will be unchanged whereas leaf element 510 willbe set to “pentatonic” and the data value of leaf element 512 will beset to “major”. As a result when the agent is applied, for exampleextracted by the processor for operation under control of the musicalinstrument, the revised scale information will be implemented and theinstrument will automatically play in the major pentatonic scale of thepredetermined key. As a result a simple and easily updatable datastructure is provided.

The node numbered ‘255’ is the node reserved for descriptiveinformation, and contains inter alia the name of the node, for examplein the case of a keyboard agent “Jim's keyboard”.

In an embodiment, where the node 500 is named ‘keyboard agent’ itcontains nodes 502 for each musical instrument key. These in turn arenamed ‘key 1, key 2’ etc in the same way. Each key subtree in turncarries nodes 508, 510 512 for each of the pressure, roll and yaw valueswhich the key hardware determines. We call these more organized subtrees504 ‘atoms’. In an alternative approach a key is not representeddiscretely in a tree but is a logical entity, the key sound beingcreated based on the relevant agent's description of the associationbetween a physical key and a corresponding event associated withactuation of a key or key group output from the keyboard. In particular,the keyboard has a key object containing as many logical keys asrequired. The language agent then associates, for example, theidentification by the user of “key 1” with the appropriate logicaloutput of the keyboard.

Each key also carries a node called ‘light’ 514 which is an input node.The value here is the color and intensity information for the lightattached to the key. This can be set by a network message, lighting thelight.

Each agent is visible over the network and can link to every otheragent. A significant facility is the ability to connect atoms together.This is done by setting a certain leaf node (for example node 516) inthe descriptive ‘255’ subtree to the name of another atom. That tellsthe atom to synchronize with the data source over the network and usethe value as its input. So for example, the “light” atom may beconnected to some controller 518 and the key light will change valueaccordingly. Atoms may take one or multiple inputs as appropriate andsimilarly have multiple outputs.

More than one atom may listen to a given output, as the arrangement is amulticast model, so making a connection between atoms is akin to‘tuning’ the listener into the correct source. In FIG. 14 a the keylight node is connected to an output from a light controller agent 518.The light could of course be connected to any appropriate action such asa pressure signal from a key, which would cause the light to changecolor as the user pressed the key.

Some agents are capable of analyzing all the descriptive data andpresenting an interface to the user as shown in FIG. 14 b. The languageagent 530 is one such, the GUI (designated generally 532) is another.The GUI 532 can interpret the links or connections between all thevarious elements and show them graphically at 536. The connection viewcan show all or selected agents and the corresponding connectionsdependent on space requirements. The “target” agent can be located atthe center and highlighted by coloring or otherwise, with directlyconnected agents also highlighted, for example in a similar color.Arrows on the connection lines can show the direction of informationflow and agents which are indirectly connected to the target agent, orunconnected, can be grouped or demoted appropriately. The language agent530 looks at the relationships between elements and allows them to becontrolled through a natural language interface.

There may of course be many more informational elements in the 255subtree than shown. There are fields which encode relationships andwhich describe actions which the agent can perform. For instance, asample player provides actions to load samples from a sample library.These are used by the language agent to allow the user to perform moresophisticated actions than simply setting a value.

Hence the system is more extensible. The language agent 530 knows how tointerpret keypresses as language input, and how to find various elementsof the system based on their description. The actual actions are sent tothe agents 534 which know how to perform them. Simply adding anotheragent to the network can expand the vocabulary of the system, and addnew actions which can be performed, because the language agent retrievesall that from the new agent.

The ‘natural’ language capability derives from the language agent'sability to see the global picture in this way. It can look at therelationships between agents and allow the user to name things byrelationship. For example, if a saxophone agent (called ‘sax’) isconnected to a mixer agent (called ‘mixer 17’) by virtue of its treestructure, and the mixer has a volume control, then the user can say‘sax volume to 11 set’ rather than having to remember that the sax isconnected to mixer 17 and saying ‘mixer 17 volume to 11 set’. The datastorage structure can be managed by a state manager (which can beanother agent) which can store and retrieve states at any time.

Of course any number and variety of agents performing or controlling anyappropriate operation or element can be provided.

A scaler agent has an input called ‘scale.’ The language agent sees thisinput, and so allows the user to say for example ‘scale to pentatonicmajor set’ which it translates into an action, in this case simplysetting the scale value.

A further agent is the “talker” agent. This provides the ability to bindentire phrases or sentences to individual keys in such a manner that acommand is already parsed and partially executed. This means thatcomplex commands can be set by the musician and then executed inperformance with ease and in a musically timely manner providing anunprecedented amount of control over the software environment whilstactually in performance.

To use the talker, a talker is created and a key group connected to it.The talker then provides a means to associate a language phrase with anykey in that group. As is normal for key groups, there is a bidirectionalconnection between group and talker, the backward direction carriesstatus information used to drive the key LEDs, lighting any keys whichhas phrases associated.

The talker is in itself a simple agent. It exploits the ability of thelanguage agent (and the rest of the natural language system) to ‘predigest’ language phrases and ready them for deferred execution. Thetalker itself merely provides the trigger by which these phrases areexecuted.

The advantage of the talker architecture is that if it is possible foran operation to be performed ‘instantly’ then that will happen. Therewill be no delay for the language agent to parse and analyse the phrasewhen the key is pressed. Indeed, the language agent may be takenentirely out of the loop.

As an example, we can construct the phrase ‘piano to pentatonic majortune’ which will change the tuning of the piano to the pentatonic majorscale. To associate it with talker's key 1, we say instead ‘piano topentatonic major when 1 tune’. The language agent splits this up into 2parts. It requests that the piano agent create a ‘deferred executionobject’ corresponding to ‘piano to pentatonic major tune’ and alsorequests that the talker creates a data source which will carry animpulse when key 1 is pressed. It then connects these together. Thetalker causes the corresponding key to light to indicate that it carriesan action.

When the key is pressed, the impulse goes straight from the talker tothe piano. This may happen completely in a part of the system optimizedfor low latency, ensuring the latency between key press and action is aslow as possible. There is no delay for the language agent to execute thesentence.

This is an instance of a more general mechanism. All verbs in the systemmay be used to create “deferred execution” objects. The natural languagehas a mechanism by which agents like the talker can advertise theability to extend the syntax of the language. The idea is to do as muchof the processing ‘up front’ and leave as little as possible to do atexecution time. For many simple parameter changes, it's possible toreduce the amount of work that needs to be done to operations that canbe carried out with little work, and thus may be performed in the partof the system optimized for low latency.

As seen in the example above, the talker provides the extension ‘whenkey <number>’ is pressed. For more complex, e.g., non key-press driven,scheduling, the talker agent may invoke a scheduler agent. The schedulerprovides a similar extension describing for example musical time,allowing phrases such as: ‘piano to pentatonic major at bar 16 tune’ and‘piano to pentatonic minor at bar 1 tune’. Together these would tune thepiano at the beginning of the song and again at bar 16. ‘Drummer volumeto 0 at bar 0 set’ ‘drummer volume every 2 bar until bar 20 up’. Thiswould silence the drummer at the beginning of the song, and thenincrease the volume every 2 bars until bar 20.

The mechanism here is identical. The phrases are split into deferredobject and trigger. The scheduler here provides the trigger impulses asthe song clock runs. Other extensions could be written to providetriggers which will fire in almost any conceivable set of circumstances.

The basic manner in which words can be entered, phrases determined andimplemented can be further understood with reference to FIG. 18.Internal operation of the language agent can be further understood fromthe flow diagram shown in FIG. 18. The process starts at step 800 wherea list of phrases is cleared at step 802. At step 804 the process waitsfor the next whole word and if at step 806 it is the start of the nextphrase (for example identified by certain words such as “as, if, when”)then, if at step 810 the next whole word is a verb then, at step 812,the set of action combinations that matches the phrases is found and runand the list of phrases is cleared again at step 802. The process thenreturns to a new phrase start at step 804.

The verb hence acts as a sentence delimiter, which may be composed ofmultiple phrases. Agents can expose verb descriptions in their metadatawhich the language agent reads. These descriptions include tests todetermine which phrases are appropriate for each action. As discussedabove in relation to the talker agent, verbs can be invoked directly, orcan be used to create deferred action objects for later invocation.These deferred actions are connected to trigger objects and the actiontakes place each time the trigger activates. Some agents thus exposedescriptions of “trigger objects” that they can create. For example thetalker agent described above creates trigger objects which trigger on asimple key press, and the scheduler agent creates trigger objects thatgo off at described times for example at certain points defined in amusical performance. Both trigger and deferred action are normal atomsin the state and the connection between them is a normal atom-atomconnection.

Referring once again to FIG. 18, at step 806 if the word is not thestart of a phrase then the word is added to the current phrase at step814. If the word is not a verb at step 810 then a new phrase is startedat step 815 and the process returns to step 804.

Referring to FIG. 19 phrases including both immediate and deferredexecution objects are shown. For example phrase 816 includes words “alldrummer” 816 a and a delimiting verb “play” 816 b which, when programmedby inputting the correct key sequence, will invoke a play action on alldrummers. Phrase 818 again includes the words “all drummer” 818 a,deferred execution object “when 1” 818 b and the delimiter verb “play”818 c which creates a deferred play action on all drummers creating atrigger object on talker key 1. Accordingly when the key correspondingto key 1 as defined on the talker is actuated this triggers the alldrummer play action. Referring to phrase 820 the words “all volume” 820a “to zero” 820 b, “when to” 820 c and “set” 820 d creates a deferred“set to zero” action on all volumes when triggered by talker key 2.Phrase 822 specifies a more complex arrangement for example also using ascheduler agent including “piano volume” 822 a, “every two bar” 822 band “up” 822 c which creates a deferred “volume up” action on the pianowhen triggered by a trigger object created on the scheduler which willtrigger every two bars.

In order to rapidly detect changes within components or agents allowingreal time operation the nodes of the data structure including root,branch and leaf elements can further be assigned with two check values,one which is associated with the data value for the dode itself—forexample a hash of the ASCII or hexadecimal values associated with thedata value and the other built from the identity and hash values of anychildren. The leaves have a null value dependent on the hash functionindicating no children.

Accordingly internal state is represented in a form which can be easilypropagated between agents and any change in the data value will resultin a change in the hash value and the server (e.g., data storage)routinely transmits both check values for comparison by the client(e.g., a participant GUI) with stored values, allowing a rapid checkthat the values are up-to-date. So, for example, where the mode datavalue associated with leaf element 508 and the mood data valueassociated with leaf element 510 are changed but the key data valueassociated with element 506 is not changed, then the hash valuesassociated with leaf values 508 and 510 will change but not thatassociated with 506. Each branch element has a hash value associatedwith the hash values of the leaf or branch elements depending therefrommeaning that the hash values associated with branch elements 504 and 502will also change. Similarly the hash value at the root element is afunction of the hash values of each or branch element dependingtherefrom as a result of which the hash value of the root element 500will also change.

Accordingly any changes in the agent can quickly be detected by theprocessor by comparing the current hash value of the root element of thestored previous hash value. If this has changed then this means thatthere has been a change in the agent and the processor can traverse downthrough the tree assessing the hash value associated with each dependingleaf or branch value until it reaches a leaf whose hash value haschanged. In particular, if the root data check is wrong, the client hasto retrieve the root node data.

If the root tree check is wrong, this means that something has changeddeeper in the tree. The client then requests the data and tree checksfor all the direct children of the root node. By comparing these withstored values, the client can see which paths down the tree are wrong.It then applies this same process recursively to each of the childrenthat have the incorrect checks. Thus the client can retrieve only theparts of the tree that have changed, avoiding the need to search throughthe whole tree.

As a result the processor can quickly update implementation of the agentin real time such that all changes are quickly and efficiently detectedand implemented and the state of an agent can be simply and quickly readfrom a remote location allowing synchronization with it. The processrelies on interaction between the data storage/server and the observeror client to compare live and stored values. It will be noted that aserver may serve multiple clients, for example multiple participants toa performance but is stateless with respect to the clients such that itis not necessary to maintain individual records per client.

The manner in which the interface can be used for both performance andcommand systems can be further understood with reference to FIG. 15. Atstep 600 a user input is received for example by depression of a key onthe instrument. At step 602 the input is processed at themechanical-electronic interface. At step 604 it is processed by thesystem where performance and control or command are separate systems asdiscussed in more detail below. In the case of the performance system,at step 608 the system determines the key pressure and also any roll oryaw values in the manner discussed above. This value is then provided tothe processor via the electronics processor interface and the processorthen performs the operation corresponding to these values.

For example the key selected may determine the note played, the pressuremay determine the volume, the roll value may determine the vibrato andthe yaw value may determine any bending or change in the pitch of thenote. The key pressure may be interpreted according to whatever criteriahas been set up such that it may change the voice of the instrument,control another instrument, control other performance affects such aslighting and so forth, governed only by the processing capability of theprocessor and the parameters set by the performer or user.

In one aspect the tuning is derived from the key by a specialized‘tuning agent’ called the scaler. There is a processing pipeline madefrom a number of agents connected together. Keys are connected to KeyGroups which build a customized group of keys (which can even be akeyboard), and take a number of keys and aggregate their signalstogether. An individual key has pressure, roll and yaw outputs, and akey group has single pressure, roll, and yaw outputs which contain thecorresponding key signals grouped together. This is an importantproperty of the system, allowing multiple values to be carried togetherunder one name. It also presents a ‘key number’ output which encodes thekey position in the group.

The output of a key group includes a control stream which encodes thescale and pitch information, which is implemented by the scaler,allowing different key groups attached to the same sound generator tohave their own melodic behavior. The aggregated information is then fedto a scaler agent, and the scaler generates frequency information basedupon which key is pressed. It has a ‘pitch bend’ input, which, ifconnected to a suitable output like the roll or yaw output from a key,will cause the frequency to be modified. The input could be connected toanything else in the system. If there was a footswitch agent whichpresented the pressure on a switch as an output, that could readily beused to modify the frequency generated by a key.

The control system and performance systems are entirely separate, andrun concurrently. They are separate programs which may even be runningon different computers. A mode key is connected to an agent which simplyadjusts the routing from the keyboard to all the various agents(instruments, language, etc) which are connected to it. When the userselects the sax, the signals to all the users of the keyboard except thesax are temporarily disabled. Likewise, when language is enabled, thekey signals to everything except the language agent are suppressed. Thekey signals are routed through software switches which are controlled bythe mode key.

At the control or command system, at step 632 subsequent user inputssuch as sequential key presses are processed to identify specific keyspressed and correlate these with the agents displayed on the GUI. Forexample where the agent is identifiable and selectable by apredetermined numerical code or note sequence corresponding to keys onthe instrument or interface, then keying the relevant code will transferthe selected agents for execution. Once all of the required agents havebeen selected for example by identifying the desired natural languagecommand and loading the corresponding agent into a control processrepresentable for example as an execution line at step 634, 636, thenatural language instruction can be executed for example by pressing thecontrol key once again at step 638. For example the instructions mayhave been as set out in the example above, a change to another key. Theperformance system then allows generation of a sound output 610according to the executed instruction.

It will be seen that a simple yet highly flexible sound creationinterface is provided according to the steps set out above allowing bothan enhanced dynamic and interactive musical performance together withrapid real-time programming of the system to vary all aspects of theperformance. Creation and manipulation of sound creation components suchas agents is facilitated through the use of a GUI programmable by theinstrument itself, and a manager is provided ensuring that the invokedagents are executed in real time allowing all aspects of the performanceto be controlled and manipulated at all times.

The interface itself can have any number of keys and any function can beattributed to any key allowing it for example to by played usingfingering styles from known instruments or entirely new fingering stylesfrom, but also allowing far more complex musical concepts to beexpressed for example by attributing sequences of notes, cords or othereffects to any key as well as more general control commands allowingcontrol of other aspects of the performance such as mastering controlother instruments connected to the system or other performance aspectsuch as lighting, sound and so forth. The key structure on the interfacecan be of any complexity and can even include a single key with multipledegrees of freedom, or other user interface elements such as breathpipes, strings, valves, pedals and so forth. The sound creationinterface can take the form of the mechanical interface with built inprocessor and, as required, GUI, the combination of mechanical interfacewith an external processor and GUI or indeed can take the form of theprocessor or optionally GUI itself with any appropriate user input.

The skilled person will note that the various embodiments describedherein are combinable where appropriate and that they are intended as anon-limiting disclosure.

The invention claimed is:
 1. A user input element comprising a sensor inoperable connection therewith for detecting movement in three axes of auser input element, the sensor comprising a light emitter, a lightdetector and a shutter component arranged to be in operable connectionwith the user input element and to be moveable therewith in response touser actuation of the user input element to at least partially occlude alight path between the light emitter and light detector, whereby a levelof light detected at the light detector corresponds to a user actuationof the user input element; a resilient biasing element upon which theshutter component is supported for movement thereof in response to useractuation of the user input element; and a housing in which the sensoris housed, and a sealing element disposed in the housing between theuser input element and the sensor for prevention of ingress of any oflight, liquid or solid matter therein.
 2. A user input element asclaimed in claim 1 in which the sealing element comprises a rubbergasket.
 3. A user input element as claimed in claim 2 in which therubber gasket is manufactured from carbon loaded silicone rubber.
 4. Auser input element comprising: a sensor in operable connection therewithfor detecting movement in three axes of a user input element, the sensorcomprising a light emitter, a light detector and a shutter componentarranged to be in operable connection with the user input element and tobe moveable therewith in response to user actuation of the user inputelement to at least partially occlude a light path between the lightemitter and light detector, whereby a level of light detected at thelight detector corresponds to a user actuation of the user inputelement; a resilient biasing element upon which the shutter component issupported for movement thereof in response to user actuation of the userinput element; a prismatic element arranged to receive light emittedfrom the light emitter and to focus the light on the light detector; inwhich the prismatic element comprises a first prism element arranged toreceive light from the light emitter and to direct the light towards asecond prism element, the second prism element being arranged betweenthe first prism element and the light detector for receiving lightpassing through the first prism element and directing the light onto thelight detector; in which there is provided a plurality of light emittersarranged around the light detector, and a plurality of first prismelements corresponding to the plurality of light emitters, each of theplurality of first prism elements being arranged to receive lightemitted from a corresponding one of the plurality of light emitters andto direct the light towards the second prism element; and in which theplurality of light emitters and the light detector are disposed on aprinted circuit board for electronic communication with a processor;four light emitters arranged around a central light detector and whereinthe prismatic element comprises four first prisms, each of the firstprism elements corresponding to a respective one of the four lightemitters and being arranged around a central one of the second primelements.
 5. A sound creation interface comprising: a user input elementcomprising a sensor in operable connection therewith for detectingmovement in three axes of a user input element, the sensor comprising alight emitter, a light detector and a shutter component arranged to bein operable connection with the user input element and to be moveabletherewith in response to user actuation of the user input element to atleast partially occlude a light path between the light emitter and lightdetector, whereby a level of light detected at the light detectorcorresponds to a user actuation of the user input element; a spring uponwhich the shutter component is supported for movement thereof inresponse to user actuation of the user input element; a prismaticelement arranged to receive light emitted from the light emitter and tofocus the light on the light detector; in which the prismatic elementcomprises a first prism element arranged to receive light from the lightemitter and to direct the light towards a second prism element, thesecond prism element being arranged between the first prism element andthe light detector for receiving light passing through the first prismelement and directing the light onto the light detector; in which thereis provided a plurality of light emitters arranged around the lightdetector, and a plurality of first prism elements corresponding to theplurality of light emitters, each of the plurality of first prismelements being arranged to receive light emitted from a correspondingone of the plurality of light emitters and to direct the light towardsthe second prism element; and in which the plurality of light emittersand the light detector are disposed on a printed circuit board forelectronic communication with a processor; and wherein: a plurality ofuser input elements arranged in an array on the printed circuit board;and a base housing upon which the printed circuit board is disposed, anda rigid curved frontage comprising a laminate structure disposed overthe plurality of user input elements so as to allow user actuation ofthe plurality of user input elements.
 6. A sound creation interface asclaimed in claim 5 in which the laminate structure includes an aperturethrough which each user input element is arranged to protrude for useractuation thereof.
 7. A sound-creation interface as claimed in claim 5further comprising at least one of user input elements being inelectrical communication with the processor for detection ofunintentional external actuation thereof.